feat: automatically reap terminal crews and orphaned lifecycle state - #41
Open
ruby-dlee wants to merge 28 commits into
Open
feat: automatically reap terminal crews and orphaned lifecycle state#41ruby-dlee wants to merge 28 commits into
ruby-dlee wants to merge 28 commits into
Conversation
ruby-dlee
force-pushed
the
fm/fm-auto-reap-gc
branch
from
July 27, 2026 18:23
97a61ed to
3f0e3c5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Verify the backlog premise against current code and history, then make Firstmate continuously garbage-collect done and merged crews and recover orphaned no-mistakes or agent lifecycle state so terminal zombies cannot accumulate memory. Reaping must happen automatically from the supervision and terminal-state paths, use the ordinary guarded teardown rather than broad process-name killing, cancel only the exact matching active no-mistakes run, and fail closed on ambiguous identity, live owners, dirty or unlanded work, X-linked tasks, or uncertain Treehouse leases. Recover acquisitions stranded by an owner crash only with exact PID plus process-start proof and exact unique lease authority. Fix any teardown false-positive that prevents safe native Herdr auto-reap while retaining exact workspace, tab, pane, label, ancestry, and process identity guards. Exercise Herdr only in a named nondefault lab through the guarded helper with before and after fleet-state tripwires, never touch unrelated sessions, and explicitly state that this Firstmate tooling change requires no web-app, API, or realtime deployment.
What Changed
Risk Assessment
✅ Low: The prior Treehouse findings are resolved with fail-closed authoritative-state validation, distinct corruption diagnostics, retained recovery authority, focused regression coverage, and the required deployment declaration; no additional material source risks were found.
Testing
Completed 1 recorded test check.
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
🔧 **Review** - 2 issues found → auto-fixed (2) ✅
bin/fm-auto-reap.sh:343- The intent requires recovery to “fail closed on … uncertain Treehouse leases,” but a zero result here only means no matching lease was found amonggit worktree listentries. A leased slot absent from that list (for example, partially registered or prunable after the crash) is not checked against authoritative Treehouse pool state, yet the acquisition record is deleted as proof of no lease. Retain the record unless absence is proven directly across the relevant authoritative Treehouse state.docs/architecture.md:38- The authoritative intent requires the change to “explicitly state that this Firstmate tooling change requires no web-app, API, or realtime deployment.” The new architecture documentation describes auto-reaping but contains no such deployment statement. Add the required explicit declaration to the change documentation.🔧 Fix: Captain, fail closed on uncertain Treehouse lease absence
1 error still open:
bin/fm-treehouse-lib.sh:129- The intent requires recovery to “fail closed on … uncertain Treehouse leases,” but this absence proof accepts malformed authoritative entries: a leased entry with a missing/non-stringlease_holder(or missing/invalid path) does not raise and therefore counts as proof that the expected holder is absent. That letsrecover_acquisitiondelete its authority record despite corrupt/ambiguous Treehouse state. Validate every leased entry’s holder and path schema before treating a non-match as proven absence.🔧 Fix: Captain, reject malformed Treehouse lease absence proofs
✅ Re-checked - no issues remain.
command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; uv run --directory tools/agent-fleet --locked pytest || rc=1; uv run --directory tools/agent-fleet --locked python -m compileall -q src || rc=1; exit "$rc"🔧 Fix: Stabilize watcher handoff and lock takeover tests
1 error still open:
command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; uv run --directory tools/agent-fleet --locked pytest || rc=1; uv run --directory tools/agent-fleet --locked python -m compileall -q src || rc=1; exit "$rc"🔧 Fix: Isolate account recovery fixtures from concurrent runs
1 error still open:
command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; uv run --directory tools/agent-fleet --locked pytest || rc=1; uv run --directory tools/agent-fleet --locked python -m compileall -q src || rc=1; exit "$rc"🔧 Fix: Fix generation-scoped task temp ownership
1 error still open:
command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; uv run --directory tools/agent-fleet --locked pytest || rc=1; uv run --directory tools/agent-fleet --locked python -m compileall -q src || rc=1; exit "$rc"🔧 Fix: Waiting for overlapping full suite before clean verification
1 error still open:
command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; uv run --directory tools/agent-fleet --locked pytest || rc=1; uv run --directory tools/agent-fleet --locked python -m compileall -q src || rc=1; exit "$rc"✅ **Document** - passed
✅ No issues found.
🔧 **Lint** - 1 issue found → auto-fixed ✅
🔧 Fix: Fix ShellCheck ownership and test-script warnings, captain
✅ Re-checked - no issues remain.
✅ **Push** - passed
✅ No issues found.